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Abstract 

This paper introduces ipyclam, a new way of manip¬ 
ulating networks in CLAM (C-|—(- Library for Audio 
and Music) by using the Python language. This ex¬ 
tends the power of the framework in many ways. 
Some of them are exploring and manipulating live 
processing networks via interactive Python shells, or 
extending the power of visual prototyping in CLAM 
by adding complex application logic and user in¬ 
terfaces with PyQt/PySide. The described Python 
API, ipyclam, by redefining the engine layer, can be 
reused to control other patching based systems such 
as JACK, gAlan... 
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1 Introduction 

CLAM (C++ Library for Audio and Music) 1 
is a free software framework to develop ad¬ 
vanced signal processing systems [Amatriain et 
al., 2007]. Some successful use cases include 
instruments [Haas, 2001; Mann et al., 2007], 
voice processing [Sommavilla et al., 2007], audio 
and music information retrieval [Gomez, 2006; 
Gouyon, 2005; Amatriain et al., 2005; Ong, 
2007], and 3D audio [Arurni et al., 2009; Giulio 
Cengarle, 2012]. 

As its name states, CLAM is a C++ frame¬ 
work. General purpose dynamic languages, 
such as Python, do not mix well with real-time 
audio programming. Those languages hide as¬ 
pects that are important to control in real-time 
programming, for example, memory manage¬ 
ment and operations that imply system calls 
that could stall the real-time thread. But real¬ 
time restrictions only apply to the processing 
code. Properly designed audio software sepa¬ 
rates the real-time code from the rest where 
those restrictions does not apply: setup, user 
interface, application logic... CLAM fosters a 

nttp://clam-project.org 


programming style which clearly localizes real¬ 
time code. For the remaining code without real¬ 
time restrictions, Python may still have an in¬ 
teresting role to play. 

This paper introduces ipyclam, a new way of 
manipulating CLAM data flow definitions ( net¬ 
works ) by using the Python language. This 
extends the power of the framework in many 
ways. For example, it can be used to build 
complex networks, like the one shown in Fig¬ 
ure 1, that are hard to build by graphical means. 
Those manipulations could be done interac¬ 
tively, by integrating interactive Python shells 
like IPython [Perez and Granger, 2007], into 
the CLAM patching tool, the NetworkEditor. 
And last but not least, it extends CLAM graphi¬ 
cal prototyping architecture, currently based on 
graphical design tools that generate fixed data 
flow and single dialog interfaces. With Python 
we can add rich application logic and interfaces 
based on PySide [Bert, 2012] or PyQt [Summer- 
field, 2007] without raising the difficulty to the 
point of requiring C++ development. 



Figure 1: Complex networks are hard to design 
by pointing and clicking. 

The rest of this paper has the following struc¬ 
ture: Key concepts of the CLAM framework are 
introduced in section 2. Section 3 describes the 
new Python API at user level. Section 4 ex¬ 
plains the internal design and how it enables 





















the reuse of the user API for other patching sys¬ 
tems. Section 5 explains how to build PyQt/Py- 
Side interfaces that can be related to CLAM 
networks and how all that leads to a more pow¬ 
erful prototyping architecture. Finally, section 
6 evaluates the already reached milestones and 
the ones that are at reach from now on. 

2 CLAM elements 

This section will shortly introduce the basic 
components of the CLAM framework needed to 
understand this paper. A more insightful de¬ 
scription can be found in the referred literature 
about CLAM. 



Figure 2: A processing unit 

Audio processing is modularized into objects 
called processing units according the CLAM 
nreta-model [Amatriain, 2005]. A processing 
unit consumes and produces data tokens by its 
input and output connectors. Connectors are 
called ports when data flow is continuous and 
they are called controls when data is sent or re¬ 
ceived unevenly. Token data can be any C-H- 
type but each connector is bound to a single 
type. When connecting connectors of different 
processing units, they must be complementary 
(input and output), same kind (port or control), 
and same type (data token C+-1- class). 

Each processing unit has a set of structured 
configuration parameters. Configuration and 
connection is done before run-time so that any 
operation that requires resource allocation can 
be done outside the real-time thread. 

A network is a set of interconnected process¬ 
ing units. The network schedules the execution 
of the units under a given audio back-end (Por- 
tAudio, JACK, LADSPA, LV2, VST...). Back¬ 
end data is fed from and to special units in¬ 
side the network called sources and sinks. Then 


the network topology mandates the data-flow 
scheduling [Armin', 2009]. 

UI binders are used to relate a CLAM net¬ 
work to a user interface, currently Qt, but not 
restricted to it. The programmer can establish 
such relation by defining custom properties on 
the elements of the user interface (widgets). UI 
binders detect such properties and add any re¬ 
quired stuff to bind them to the network. Com¬ 
mon examples of UI binders are the ones used 
to bind user interface for playback control and 
monitoring, processing unit configuration, data 
token visualization, and user control sending. 



Figure 3: Visual prototyping architecture 

All those elements enable the CLAM visual 
prototyping architecture [Garcia, 2007] illus¬ 
trated in Figure 3. Both the processing net¬ 
work and the user interface can be designed with 
graphical tools, CLAM NetworkEditor and Qt 
Designer respectively. Both can be stored as 
XML, loaded later in run-time, and related by 
applying the binders. A tool called Prototyper 
does that by taking the XML files by command 
line. 

Most elements in this architecture (process¬ 
ing units, token data type handlers, back-ends, 
UI binders, widgets...) can be extended via plu¬ 
gins. Most of those extendible objects are avail¬ 
able through abstract interfaces and factories. 
























































































3 ipyclam user API 

3.1 Goals 

ipyclam’s main goal is providing the API to be 
able to build and explore a CLAM network. 
Defining the processing code inside processing 
units is reserved to C++ code to fit real-time 
constraints. 

An explicit choice has been taken on not de¬ 
signing Python API as a direct map of the 
C++ CLAM API, but to make it conveniently 
Pythonic. Direct C++ library mapping often 
leads to a badly designed Python interface. De¬ 
sign decisions taken in C++ API just because 
of C++ idiosyncrasy, may get pointlessly repli¬ 
cated in Python, and opportunities of using 
Python features such as attributes, iterators, 
generators or dynamic interface creation, may 
get lost. 

Another choice is to provide a powerful tab 
completion for interactive Python shell. It is 
not just about discovering the static API but 
taking a step further and discovering the run¬ 
time structure of the objects via tab completion. 
Because of that, such structure should be avail¬ 
able as completable attributes. 

Convenient ways of expressing things are fa¬ 
vored but whenever those convenient ways are 
not expressive enough to express everything, in¬ 
stead of discarding the convenient one, we add 
the less convenient one as alternative. 

For example, processing units are accessible 
as network attributes with their names. Also 
processing ports, controls and configuration pa¬ 
rameters are accessible as processing attributes 
as well. That interface is compact and conve¬ 
nient when doing tab completion. 

net.Sink.Audio 

But this is not a general solution. Many 
names are not valid identifiers. Subscript ac¬ 
cessors are provided to solve those cases: 

net ["A processing"] ["1"] 

Still, you may find two subelements of differ¬ 
ent kind with the same name, or with a name 
that matches an actual attribute or method of 
the object. For those cases, it is useful to pro¬ 
vide scoping attributes. So the syntax that will 
always work would be: 

net.processings [" Sink "] . inports ["Audio"] 

But this is more verbose than the first pro¬ 
posal. It is a design choice when this kind of 


situation appears, to provide both the conve¬ 
nient and the complete options so we get the 
best of them. 

3.2 An example 

This is a minimal example that creates a 3 chan¬ 
nel cable, that just copies the input to the out¬ 
put, and plays it under JACK: 

from ipyclam import Network, time 
net = Network () 

# creating units 

net.source = "AudioSource" 
net.sink = net.types.AudioSink 

# configuring 

net.source.NSources = 3 
net.sink.NSinks = 3 

# connecting 

net.source > net.sink 

# Playing as JACK client for 1 minute 
net.backend = "JACK" 

net.play () 
time.sleep (60) 
net.stop () 

3.3 Creating processing units 

Notice that the first processing in the example, 
source, is created by assigning a string, the pro¬ 
cessing type name, to a new attribute with the 
name of the processing. The second one, sink, is 
created instead by using the ’net.types’ object. 
Such object is convenient for interactive use to 
discover the available types by tab completion. 

>> net.types.Audio [tab] 

AudioSink , AudioMixer , AudioSource 

If the unit name is not a proper Python iden¬ 
tifier, the subscript syntax can be used as well: 

net ["My Sink"] = net.types.AudioSink 

3.4 Configuring 

Configuration parameters can be accessed di¬ 
rectly as direct attribute or subscript of the pro¬ 
cessing unit. They can be accessed as well inside 
the scoping attribute config to avoid conflicts 
with other processing subelements or common 
attributes and methods. 

net . source . conf ig . NSources = 3 

Every time a parameter is set, the object is 
reconfigured, but reconfiguration may be an ex¬ 
pensive process. To address that issue reconfig¬ 
uration may be held while setting a set of pa¬ 
rameters for a given unit using the with state¬ 
ment: 

with net.mymodule.config as c : 
c.AParameter = "A Value" 
c.AnotherParameter = 23.2 


Configuration parameters are typed, type 
checking is done on assignment rising TypeEr- 
ror if the type is not the proper one. In CLAM, 
configuration parameters can be instantiated or 
not. In Python uninstantiated state is repre¬ 
sented by the None value. 

Some configurations are structured using pa¬ 
rameters that are configurations themselves. 
Such sub-configurations can be accessed as nat¬ 
ural by accessing successive attributes. 

net.mymodule.SubConfig.Paraml = 4 


3.5 Connecting 

The example uses the greater-than operator to 
establish the connection. Both sides of the op¬ 
erator refer to the processing units, but indeed 
what gets connected are the connectors. So this 
is a short-cut for connecting each port pair-wise: 

net.source ["1"] > net.sink [" 1" ] 

net.source ["2"] > net.sink [ " 2 " ] 

net.source["3"] > net.sink [ "3 " ] 

Or, generally, by using the iterators of inports 
and outports attributes: 


CLAM Network Editor - Untitled [modified] 

File Network Interface View Help 
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Processing Toolbox * * 



Processing Toolbox Description 


I In 119] 
I In [20] 
| In [21] 
I In [22] 
I In [23] 
In [24] 


sink=" AudioSink" 
sou rce=" AudioSou rc e" 
sink.NSinks = 4 
source .NSources = 4 
source ._outports[:] : 


:.sink._inports[:: -1] 


Figure 4: IPython console integrated in the Net- 
workEditor interface 

net = Network () 

net. load("mynetwork .clamnetwork" ) 

net.save ("mynetwork-copy.clamnetwork") 

Indeed you can get the XML string for the 
current network using xml(). 
print net.xml () 


for inport, outport in zip( 
net.source.outports , 
net.sink.inports , 

) : 

inport > outport 

Similar iteration can be done with incon¬ 
trols and outcontrols processing unit attributes. 
They can be used as well with Python slices. 
For example, if we want to reverse the channels: 

net.source > net.sink.inports [: :-1] 

Or first and third to the first two: 

net.source.outports [: :2] > \ 

net.sink.inports [ :2] 

3.6 Playback control 

The audio back-end can be set by assigning the 
backend special network attribute. For exam¬ 
ple, if we wanted to use the PortAudio audio 
backend we could use: 

net.backend = "PortAudio" 

The network has several methods, pause(), 
playQ and stop(), to control the playback, and 
several methods, isPlaying(), isPausedf) and is- 
Stopped(), to query the playback status. 

3.7 Serialization 

Networks can be loaded from XML files gener¬ 
ated by NetworkEditor. 


Although XML is somehow readable, in fact, 
we found that Python code is even more read¬ 
able than XML. An ipyclam network is able to 
generate code to reconstruct itself. 

>> print net.code (" mynet" ) 
mynet = Network() 
mynet.sink = "AudioSink" 
nynet . sink . NSinks = 3 

This feature is quite powerful. Given a static 
network stored as XML, it can be converted 
to Python code and as Python code it can be 
parametrized or turned into a more smart pro¬ 
gram. 

This also opens the door to the use of Python 
code as serialization format instead of XML. In¬ 
deed Python code using ipyclam API is more 
compact and readable than XML. Despite that, 
deprecating XML is not yet an option as it is 
not save to use Python interpreter as parser. A 
Python interpreter will allow to execute more 
than just network definitions. 

4 Implementation 

This section gives a slight overview on how ipy¬ 
clam API has been internally implemented and 
how this design allows extending the use of the 
API to control other patching systems. 

ipyclam is designed in two layers as shown in 
Figure 5. The user layer is the one that provides 
















Figure 5: Two layers architecture 

the API explained on previous sections, with all 
the sugar for the many redundant and pythonic 
ways of expressing the same operation. 

But that layer is stateless. In order to per¬ 
form the actual operations it relies on a engine 
layer which holds the actual state, in this case, 
the C++ CLAM Network object. Those many 
ways of performing a given operation at the user 
API converge in a single entry point at the en¬ 
gine layer resulting in a narrower API at that 
level. 

This design in two layers strengthens the re¬ 
liability of the implementation. The user API 
can be developed ignoring all the complexities of 
the adapters to the C++ CLAM code by pro¬ 
viding a mock-up engine in pure Python. A 
narrow engine API reduces the number of oper¬ 
ations to test for the engine and centralizes the 
state checks for the front-end testing. A state¬ 
less front-end avoids errors on the bookkeeping 
of duplicated information. 

Another positive side effect of this design is 
that this narrow engine API can be reimple¬ 
mented to address any other patch like systems, 
such as JACK, Patchage, gAlan... As result all 
the rich ipyclam API interface can be reused for 
those systems. Other patching programs can 
integrate the Qt console like the one that now 
NetworkEditor has and is shown in Figure 4. 

5 Prototyping user interfaces 

CLAM visual prototyping architecture, ex¬ 
plained in section 2, provided a way to build 
a simple audio application by joining two parts 
designed visually: a CLAM network and a Qt 
Designer interface. Although that architecture 
generated decent applications, it has a clear ceil¬ 
ing of what you can build. Applications are 
limited to simple application logic, a single di¬ 
alog and a fixed processing data-flow. If any¬ 
one wants to go beyond that, C++ program¬ 


ming skills are required, so the learning thresh¬ 
old goes up and the development work-flow gets 
harder and slower [Garcia, 2007]. 

An intermediate solution is to introduce 
Python as programming language for the user 
interface and application logic. Python is easier 
to learn and has a faster development work-flow. 
This section explains some features of ipyclam 
that facilitate building such applications and 
shows some examples that illustrate the scope 
of what you can do. 

5.1 PyQt4 and PySide 

Two Python bindings are available for Qt: 
PyQt4 2 and PySide 3 . Each one uses a differ¬ 
ent binding generator technology: PyQt4 uses 
SIP while PySide uses Shiboken. The resulting 
Python APIs are mostly identical, so writing 
Python code that works for either is not hard, 
ipyclam supports both. In the following exam¬ 
ples, PyQt4 is used but using PySide is just a 
matter of changing the import lines. 

5.2 A Python based Prototyper 

The following Python code provides a simplified 
version of Prototyper. 

import ipyclam , sys 
from PyQt4 import QtGui 
import ipyclam.ui.PyQt4 as ui 

# network setup 
net = Network () 

net.backend = "JACK" 
net . load (sys . argv [1] ) 

# ui setup 

app = QtGui.QApplication ([]) 
w = ui . loadUi ( sys . argv [2] ) 
net.bindUi(w) 

# run 

w.show ( ) 
net.play () 
app.exec. () 
net.stop () 

The interesting bits are the loadUi function 
from the ipyclam.ui ,PyQt4 module and the 
bindUi method of the network. The loadUi 
function is a helper that instantiates a Qt De¬ 
signer file. The bindUi method applies all the 
available binders to the user interface. Possible 
bindings are searched recursively so you can use 
it with a full interface as well as a single widget. 

This snippet has the same restrictions as Pro¬ 
totyper: It is general but it is limited to a single 
processing data flow and a single interface with 
no application logic. 

2 http: / / www.riverbankcomputing.com / software / pyqt 

3 http: / / www.pyside.org 












The good news is that now we can 
change that code to modify the network with 
the ipyclam API exposed on previous ver¬ 
sions and modify the interface with regular 
PyQt4/PySide API. 

5.3 Building interfaces from scratch 

A counterexample would be building the pro¬ 
cessing network and the interface without XML 
files, that is, using ipyclam and PyQt4/PySide 
APIs. A problem with this approach is that 
some useful audio widgets provided by CLAM 
as Qt plugins have no specific Python wrappers. 
Providing such wrappers would imply to gener¬ 
ate them for SIP and Shiboken for each spe¬ 
cific widget class in the plugin. Instead, ipy¬ 
clam provides a helper method to access the Qt 
widget factory, which creates the widgets from 
the class name string. Factory created widgets 
are handled by the generic QWidget interface, 
which includes composing them and accessing 
their properties. 

The following example implements an oscillo¬ 
scope, by binding a CLAM Oscilloscope widget 
with an AudioSource. 

import ipyclam , sys 
from PyQt4 import QtGui 
import ipyclam.ui.PyQt4 as ui 

# network setup 
net = Network () 
net.backend = "JACK" 

net.source = net.types.AudioSource 

# ui setup 

app = QtGui.QApplicat ion([]) 
w = ui.createWidget(" Oscilloscope") 
w. setPropertyC'lineColor" , "red") 
w.setProperty( 

"clamOutPort" , "source.1") 

net.bindUi(w) 

# run 

w.show ( ) 
net.play () 
app.exec. () 
net.stop () 

This example accesses specific behaviour of 
the Oscilloscope, the UneColor , by using the 
generic property interface. The same method is 
used to set the binding property clamOutPort 
that in a visually designed prototype should 
have been defined with Qt Designer. 

5.4 Hybrid approaches 

Any combined approach is feasible. Figure 6 
shows an example that comes with ipyclam that 
combines a Qt Designer file with a coded inter¬ 
face. Indeed, this example has some application 
logic not available with simple visual prototyp¬ 
ing. Notice that the combo box is filled with 



Figure 6: Extending with Python an existing 
visual prototype that uses Spectral Modeling 
Synthesis for a two voices transposition. The 
extension provides detailed configuration of ev¬ 
ery unit. 

information, the names of the processing units, 
taken from the network with the ipyclam API. 
The configure button, instead of being a bound 
widget, activates a function that takes the cur¬ 
rently selected processing unit, and launches a 
configuration dialog bound to the given process¬ 
ing configuration. 

6 Conclusions 

The API presented in this paper offers a 
new way of developing real-time audio appli¬ 
cations by combining the power and flexibility 
of CLAM, Qt and Python. The API has been 
designed with a strong stress on convenience 
and expressiveness which results in very read¬ 
able and compact code. 

An interactive Python console has been inte¬ 
grated with the graphical patching tool. This 
enables the user to build complex networks 
by interactive programming, and having visual 
feedback of the results. This work can be eas¬ 
ily extended to other patching systems just by 
implementing a narrow API. 

Indeed, a promising engine to implement in 
the future is one relying on JACK because 
CLAM users are likely to be interested in 
controlling JACK application interconnections 























from the console, just as they control inner 
units. 

Another work to be done is providing some 
useful examples built with ipyclam that give po¬ 
tential users a clear idea of the horizons of the 
platform. They also will help to mature the API 
highlighting any unpolished edges left. 

Right now, the platform excludes Python for 
processing tasks. But Python has a nice collec¬ 
tion of numerical libraries based on the numpy 
package [Ascher et al., 1999]. They could be 
used for processing algorithms for off-line pro¬ 
cessing or situations where lesser real-time con¬ 
ditions are required. Two approaches are being 
considered. One is being able to implement pro¬ 
cessing units in Python. The other is a Python 
audio back-end where Python code feeds the 
network with numpy arrays as audio input and 
output. 
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